Learning from PISA

The great competition is over, the judges have given their marks and the chalk dust has settled over the arena. A few hurt prides pick themselves up off the floor, and set about the painful task of finding out what went wrong and why they did not fare as well as they expected. This is not the aftermath of the winter Olympics, rather the results of the PISA study, the Programme for International Student Assessment. The largest world-wide test of student performance ever conducted, it provides a much needed assessment of how well various countries educate their children. And PISA served up a can of worms to some diners who can afford to pay for duck à l’orange, while other less wealthy guests came away substantially more satisfied. Many of the fatter countries had quietly been intellectually malnourishing their most important resource: school children. Investment in basic requirements is important, but what PISA revealed is that money is certainly not everything in education. It is other important factors, notably student motivation, teacher training, the role of parents and a well-established infrastructure, that have put countries such as Finland and (South) Korea at the top of the list. For other countries, many of whom spend more money per student, PISA has been a rude awakening to the urgent need for change. In Germany it has stirred an intense debate—aside from the mutual finger-pointing that usually follows bad results—as to how to achieve a long overdue and radical overhaul of the country’s education system. Barely a day passes without a press release concerning ‘reforms’, ‘improvements’ and ‘opportunities’ in education. To appreciate the significance of PISA, it pays to know how the study was conducted. More than 250 000 15-year-olds in 32 countries took a 2 h paper and pencil test comprising 45% multiple choice questions, 45% requiring self-generated answers and 10% limited range answers. It was supplemented by a student questionnaire and a questionnaire to be completed by school principals. In reading literacy, the PISA study defines five levels of proficiency that range from identifying one obvious piece of information in a text (level 1) to extracting deeply embedded information, evaluating it and inferring a conclusion (level 5). Worryingly, 10% or more of the students in some European countries failed to reach even level 1. Scores in mathematics and science are given according to three levels of proficiency, ranging from simply applying common knowledge (lowest), to the ability to relate given data and evaluate different theories (highest). Performances in a particular exercise are scored on a scale of 0–800 points depending on the difficulty of the question and the quality of the answer given. Example of a science exercise set in PISA, taken from PISA 2000, OECD (texts and questions are abridged from the originals)